Repository navigation
🧩 SyGra is now embeddable: run it as a library inside your own app - #132
Open
psriramsnc wants to merge 19 commits into
Open
psriramsnc wants to merge 19 commits into
psriramsnc wants to merge 19 commits into
Conversation
Add an extension point so applications embedding SyGra can serve a model's calls with their own client (gateway, in-house SDK, caching/recording layer, or a test double) instead of the built-in HTTP/LiteLLM clients. - LLMProvider protocol: async generate(LLMRequest) -> ModelResponse - LLMRequest: OpenAI-style messages, model, parameters, tools, tool_choice, pass-through provider_options - ExternalModel (model_type: external) delegates to a named provider; the provider owns transport/retries, so SyGra defaults to no retries/no delay - Provider registry: process-wide register_llm_provider(), context-scoped use_llm_providers(), and Workflow.llm_provider() scoped to one run - ModelResponse: optional usage (TokenUsage, with estimated flag) and served_model - ModelFactory.register_model_type() public API for custom model types - FakeProvider test double, docs page, tests
The server-down check (repeated 404/5xx responses within a short window) called sys.exit(), which kills any application embedding SyGra. Make the action configurable and let library runs raise instead, so the host decides. - New server_down_action: "exit" (default, CLI behavior unchanged), "raise" (ModelServerDownError) or "ignore" - Resolution: model config key server_down_action, then a context-scoped server_down_action(...) context manager, then "exit" - Workflow.run() uses "raise" and re-raises ModelServerDownError unwrapped - The error aborts the run instead of being swallowed per record: in-flight records are cancelled and graph post-processors are skipped - Applies to both custom models and LangGraph chat models - ModelServerDownError (a ModelError) and server_down_action exported from sygra; docs and tests
Workflow.run() calls asyncio.run(), which fails inside an already running event loop (async web apps, job workers, notebooks). Add an async path so SyGra can be embedded in those hosts. - Workflow.arun(): same arguments and result as run(); records are processed as tasks on the caller's loop, so LLM providers can use loop-bound clients - run() is now a thin asyncio.run(arun()) wrapper - BaseTaskExecutor.aexecute(): execute() split into prepare / process / finalize; the blocking setup (graph build, output file) and post-processing (data quality, sinks, metadata) run in worker threads via asyncio.to_thread - Executor construction (dataset loading) also runs in a worker thread - DatasetProcessor.aprocess_and_store_results() awaits the processing loop - Docs (library guide "Async Execution") and tests
Several pieces of per-run state lived in module globals, so two workflows running at the same time in one process (e.g. two concurrent arun() calls) overwrote each other. Some per-request state lived on shared model instances, so concurrent records could see each other's values. Per run (RunContext, carried by contextvars; opened by Workflow.run/arun): - task name (utils.get_current_task()), weighted-sampler cache (utils.get_sampler_cache(), also used by resumable state), and a run-scoped metadata collector (MetadataCollector.create_isolated(); the enabled flag is inherited, and results are copied to the global collector after the run) - signal handlers: arun() never installs them; run() still does for resumable runs, only on the main thread, and restores the previous handlers - the CLI runs without a RunContext and keeps using the module globals Per request (ModelCallState, opened by track_model_request): - token usage is reported per request and attached to ModelResponse.usage; LLM nodes read it from the response instead of _last_request_usage (still set, and still honoured when a model sets it during the request) - LiteLLM tool parameters are applied per request instead of being merged into the shared generation_params; generation_params is now a copy of the model config's parameters Docs and tests
Library use loaded ./.env with override=True on every model config load, and importing sygra loaded a .env found in the working directory or any parent directory. An application embedding SyGra could have its own environment and secrets overwritten by an unrelated .env file. - load_model_config() reads os.environ only; it no longer loads .env - remove import-time load_dotenv() from the data-quality tool modules - the CLI keeps loading .env: main.py at startup (unchanged), the tools CLI (python -m sygra.tools) and the reward_score script explicitly - Workflow.run()/arun(env_file=...) loads a .env file for that run, without overriding variables that are already set; a missing file raises ConfigurationError - connection settings can be passed straight in the model config (url, auth_token), which takes precedence over SYGRA_* env vars Behavior change for library users: scripts that relied on SyGra loading .env implicitly must now pass env_file=".env" or export the variables. Docs and tests
…library runs - DatasetProcessor restores the previous SIGINT/SIGTERM handlers after processing only in library runs (Workflow.run/arun). CLI runs keep SyGra's handlers installed after processing, as before. - ResumableExecutionManager doesn't install signal or atexit handlers in library runs. They were already skipped off the main thread, and the per-run atexit registration accumulated in long-lived host processes. CLI runs are unchanged. - Regression test for both CLI and library behavior.
…s and cost
Library callers had to re-read the output file to get results, and token usage
and failures were only available from the process-wide metadata collector.
- RunResult is the list of output records (same content as the output file,
so list usage keeps working) plus:
- failed_records: FailedRecord(id, kind, error) for model errors,
graph exceptions and schema-validation rejections
- usage_by_model / usage_by_node / usage_by_record / total_usage:
UsageStats with requests, failures, tokens, estimated flag, latency
and served models
- cost_usd via an optional cost_fn(model, usage) hook (no prices shipped)
- output_file (None when the output directory was temporary)
- YAML tasks run through Workflow now also return their records
- usage and failures are collected on the run context, independent of
metadata collection; nothing is collected outside library runs, so CLI
runs are unchanged (verified: identical output and metadata)
Docs and tests
In non-resumable runs, results are written at checkpoints, which are only triggered when a record succeeds. If the last record to finish failed (or the dataset ended before num_records), successful results since the last checkpoint were never written; with fewer records than the checkpoint interval, no output file was produced at all. The processing loop now writes any remaining results at the end of the run, before graph post-processors read the output file. Resumable runs already did this and are unchanged. Runs where every record succeeds produce the same output as before (no duplicates).
Workflow.from_config(config, base_dir=None, name=None) builds a workflow
from a dict or YAML file shaped like graph_config.yaml, without needing a
tasks/ directory.
- Prompt messages can be inline text or {"file": path}. Relative prompt
files resolve against base_dir (or the YAML file's directory), never the
working directory.
- With base_dir, relative data source/sink file_path values resolve
against it.
- Output goes to a temporary directory unless output_dir is given.
- Configured source transformations are kept, and graph_properties are read
from the in-memory config through the run context instead of a task YAML.
- The input dict is not modified.
- ConfigLoader.load_and_create() now delegates to Workflow.from_config().
- Existing-task executors are built once with the config instead of twice.
CLI, YAML-task, Studio and tools behavior is unchanged: graph properties
still come from the task YAML when no library run is active.
…c API
Library users can pass pre-processors, post-processors, lambdas, edge
conditions, data transforms, output generators and graph post-processors as
Python objects (function, class or configured instance), in the builder or in
a Workflow.from_config() dict. Dotted-path strings work as before.
- Instances are used as is, so constructor state is kept. Before, the
builder turned an instance into its class path, re-created it without
arguments, and lost the state. Local functions, classes and lambdas used to
become dotted paths that could not be imported; they are now stored as
objects.
- The builder still stores module-level functions and classes as dotted
paths, so save_config() keeps writing plain YAML.
- GraphConfig and from_config copy a config's dicts and lists but share
other values, so hook objects don't need to be deep-copyable (they may
hold a lock or a client). For YAML configs this equals deepcopy.
- TransformConfig.transform accepts objects; .transformations() builds
{"transform": ...} entries, and builder runs keep the transformations they
declare.
- New public module sygra.hooks (also exported from sygra) with the hook
base classes and HOOKS_API_VERSION = 1; stability and deprecation rules
are in its docstring and the library docs.
YAML/CLI behavior is unchanged: dotted paths always resolve to callables,
which keep the old class/function handling. A CLI run of a YAML task with a
class transform, class and function lambdas, a class edge condition, an
output generator and a graph post-processor gave identical output before and
after.
Library callers can keep data in memory end to end and join outputs back to
inputs by id.
- In-memory sources: a list source (Workflow.source(rows),
DataSource.memory(rows), or {"type": "memory", "records": rows}) is read
through a new MemoryHandler instead of being written to a temporary JSON
file. The caller's dicts are not modified. Resumable runs with a memory
source raise ConfigurationError.
- Record ids: records without an id get sygra.record_id(record), the same
SHA-256 content hash as before. In library runs every record gets one,
not only when the first record lacks an id, and output records keep the
input id even when the output_map or generator doesn't map it.
- write_output=False on run()/arun() keeps output records only in the
RunResult: no output file is written and an existing one isn't deleted.
A sink, data quality, graph_post_process or resumable execution raises
ConfigurationError before the run, because they read the output file.
- ConfigurationError from a programmatic run is no longer wrapped in
ExecutionError.
CLI and YAML-task behavior is unchanged: the new behavior applies only when a
library run context is active, and the CLI keeps strict JSON hashing. A CLI
run of two YAML tasks (an output_map without id and records with only some
ids) gave identical output and metadata before and after.
…oles=...))
A node can name a role instead of a model, for example
{"role": "generator"}. Each run maps roles to models, so the same graph can
run against different model mixes without edits. SyGra only resolves names
and enforces no policy.
- Workflow.model_roles({...}) sets defaults; run()/arun(model_roles=...)
overrides entries for one run. A from_config() dict can set defaults with
a top-level model_roles key.
- A role maps to a model name or a model config dict with "name". The
node's own keys (such as parameters) are applied on top.
- ModelFactory resolves roles from the run context before creating models,
so llm, multi_llm and agent nodes all support them. An unmapped role, or a
config with both name and role, raises ConfigurationError.
- ModelConfigBuilder.validate_config accepts role configs.
The CLI is unchanged: without a library run context, model configs are not
touched. A CLI run of a lambda + weighted-sampler task gave identical
output and metadata before and after.
…rovider tasks/examples/library_pipeline/pipeline.py turns three synthetic help articles into FAQ entries. It needs no API keys or network access and writes nothing to the working directory. It shows the features for pipelines defined outside the repo's tasks/ tree: - Workflow.from_config() with prompt files resolved against base_dir; - a configured post-processor and edge condition passed as objects; - a memory source, arun(write_output=False) and joining results by id, including sygra.record_id() for a record without one; - model roles, with a second run that swaps the reviewer model; - usage by model and cost from a toy cost_fn. tests/workflow/test_library_example.py runs the example from another working directory so it keeps working. The library examples guide links to it.
psriramsnc
marked this pull request as ready for review
November 18, 2026 05:06
- requires-python is now >=3.9,<3.14,!=3.9.7, with 3.12/3.13 classifiers and black targets. - CI runs the format, lint and test jobs on Python 3.9 and 3.13 (docs are built once, on 3.9). - fastapi is widened to >=0.124.4,<1.0 and uvicorn to >=0.38,<1.0, so SyGra installs next to applications on newer FastAPI (>=0.128) and Starlette (>=1.0). - pandas is declared as >=2.2,<4.0. It was an undeclared transitive dependency; fresh installs can use pandas 3. - uv.lock is regenerated with 3.12/3.13 forks. On 3.9-3.11 every locked version is unchanged (fastapi 0.124.4, pandas 2.3.3), so the CLI, CI and Studio environments built from the lock don't change. Checked: - Full suite (1127) plus ruff, black, isort and mypy on 3.9 and 3.13 from the lock; also on 3.13 with the newest versions a fresh install picks, and with pandas 3.0.5, fastapi 0.141 and starlette 1.4. - CLI: the same YAML task (weighted sampler, transform, lambdas, edge condition, output generator, graph post-processor) on 3.11 and 3.13 gave identical output, post-processor and metadata files, apart from a random id and the recorded Python version. - Studio: 48 read-only API requests on 3.11, on 3.13 from the lock, and on 3.13 with fastapi 0.163 / starlette 1.10 returned the same status codes and bodies, apart from directory-listing order and signed Hugging Face asset URLs; no server errors. - The built wheel installs on 3.13 next to pandas 3.0.5, fastapi 0.141 and starlette 1.4 (uv pip check: all compatible), and the library example runs from outside the repo against the installed wheel. Known: pandarallel (declared, never imported by SyGra) doesn't import with pandas 3.
…3.13 - Python 3.9.25 builds exist only for Ubuntu 22.04/24.04, and ubuntu-latest is now 26.04. The 3.9 job now runs on ubuntu-24.04; the 3.13 job stays on ubuntu-latest. - fasttext-wheel 0.9.2 has no wheels for Python 3.13, and its source doesn't build with current GCC (src/args.cc uses uint64_t without including <cstdint>). It is now installed only on Python < 3.13. On 3.13 the `fasttext` module comes from fasttext 0.9.3, which is already a dependency and has the fix. On 3.9-3.12 nothing changes; uv.lock only gains environment markers. Checked in a gcc:15 container with Python 3.13.13: fasttext-wheel 0.9.2 fails with the same args.cc:468 error as CI, and fasttext 0.9.3 builds, imports and trains a model. On macOS, with Python 3.13 and no fasttext-wheel, the full suite (1127) passes and mypy is clean.
- loader.py: remove the AutoNestedDict import. It has been unused since load_and_create() delegates to Workflow.from_config(). - dataset_processor.py: register self._handle_signal directly instead of through a lambda that only forwards the call. - utils.hook_reference: comment the intentional best-effort except.
The branch ruleset on main requires a status check named 'build'. The matrix renamed the job to 'build (<python>, <os>)', so the required check never reported. Rename the matrix job to 'test' and add a 'build' job that passes only when every matrix job passed.
ubuntu-latest no longer provides Python 3.9, so setup-python would fail on the next tag. The wheel is pure Python (py3-none-any), so the build interpreter does not change the published package.
psriramsnc
force-pushed
the
scratch/embeddable_sygra
branch
from
November 18, 2026 08:18
30471b1 to
4bee5e6
Compare
pysnc 1.1.x requires certifi<2025, which conflicts with current certifi security releases. pysnc 1.2.x requires certifi>=2026.1.4,<2027. The GlideRecord, ServiceNowClient and ServiceNowPasswordGrantFlow APIs that the ServiceNow data handler uses are unchanged.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🧩 SyGra is now embeddable: run it as a library inside your own app
🧾 Summary
SyGra was built to own the process. It reads
.env, callssys.exit()when a model looks down, keeps run state in module globals, starts its own event loop, and reads graphs and writes results under the repo'stasks/tree. That works well for the CLI, but it makes SyGra hard to embed in a web service, a job worker or a notebook 🧱.This PR turns SyGra into a well-behaved library 📦. A host application can now:
awaita run inside its own event loop;The CLI, YAML tasks and Studio keep working as before. New behavior is opt-in, or only active in library runs (
Workflow.run()/arun()). Every item was checked with a side-by-sidemain.pyrun against the previous commit.LLMProvider+model_type: externalrun()fails inside a running event loopawait workflow.arun()runs on the caller's loopsys.exit()and takes the host downModelServerDownError, which the host can catch.envis loaded, with override, on every model-config loadenv_file=is opt-inRunContext; concurrent runs are isolatedRunResultwith records, failures, usage and cost;write_output=Falsetasks/YAML; hooks are dotted pathsWorkflow.from_config(dict), hook objects, model roles🛠️ Explain the features implemented:
🔌 Pluggable LLM providers
LLMProviderprotocol:async def generate(request: LLMRequest) -> ModelResponse, with amodel_type: externalmodel that delegates to a named provider.LLMRequestcarries OpenAI-style messages, the model, parameters, tools,tool_choiceand pass-throughprovider_options.Workflow.llm_provider(provider, name=...)registers a provider for one run.use_llm_providers({...})registers for a context, andregister_llm_provider()registers process-wide.ModelResponse.usage/served_model: providers report token usage, with anestimatedflag, and which model actually served the call.ModelFactory.register_model_type(): a public API for custom model types, so callers no longer editMODEL_TYPE_MAP.FakeProvider: a deterministic test double that records every request 🧪.⚙️ Safe in-process execution
async Workflow.arun(): same arguments and result asrun(). Records run on the caller's loop, and blocking setup and finalize steps run in worker threads.run()is nowasyncio.run(arun()).RunContext: each run has its own task name, metadata collector, sampler cache, usage and failures, carried bycontextvars.arun()never installs signal handlers.ModelCallState:generation_params.server_down_action(exit/raise/ignore):exit;ModelServerDownError, cancel in-flight records and still save resumable state..env: library runs read only the existing environment, andenv_file=opts in without overriding variables. The CLI still loads.envat startup.📊 Results and data in memory
RunResult: alistof output records, so code that uses it as a list keeps working. It also carries:failed_records:FailedRecord(id, kind, error);UsageStatswith requests, tokens, latency and served models;cost_usd, through your owncost_fn(SyGra ships no prices);output_file..source(rows)/DataSource.memory(rows)now read straight from memory. Before, they wrote the rows to a temp JSON file on disk 🧹, whichDataSource.memory()never deleted. Your dicts aren't modified.idgetsygra.record_id(record), the same content hash as before.ideven when theoutput_mapdrops it, so you can join outputs back to inputs 🔗.run(write_output=False): results stay in theRunResultonly; no output file is written. Features that need the file (sinks, data quality,graph_post_process, resume) raise a clearConfigurationErrorbefore the run starts.🧱 Pipelines defined in your own code
Workflow.from_config(config, base_dir=..., name=...): build a workflow from a dict (or YAML) shaped likegraph_config.yaml, with notasks/directory.{"file": "prompts/x.txt"}, resolved againstbase_dir.graph_propertiesare kept.sygra.hooks: a public, versioned hook API (HOOKS_API_VERSION = 1) with documented stability and deprecation rules, so external subclasses don't break on internal refactors.{"role": "generator"}instead of naming a model.Workflow.model_roles({...})sets defaults, andrun(model_roles=...)swaps models per run, for example to cross-check one model family with another, without editing the graph.tasks/examples/library_pipeline/pipeline.pyuses all of the above withFakeProvider. It needs no API keys and no network access.🐍 Python 3.12 / 3.13 support
requires-python = ">=3.9,<3.14,!=3.9.7", with 3.12/3.13 classifiers and black targets.fastapianduvicornare widened to<1.0, so SyGra installs next to apps on newer FastAPI/Starlette.pandasis now declared as>=2.2,<4.0; before, SyGra used it without declaring it.uv.lock: regenerated, with every locked version unchanged on 3.9–3.11.🐛 Fixes found along the way
parameters..transformations()entries were rejected, and builder runs erased configured transformations.graph_propertiesset in Python were ignored.DataSource.memory()left its temp input file behind.⚡ Performance impact (if any):
write_output=Falseskips output-file writes and the read-back.deepcopy.arun()keeps the event loop free: dataset loading and post-processing run in worker threads.🧪 How to Test the feature
Steps for reviewers to verify functionality:
make setup-dev && make test. Expect 1127 passed (93 new tests), withmake check-lintandmake check-formatclean.id, including a generatedrecord_idfor the article without one. Short answers get a reviewer verdict. It then prints usage by model, the cost, and a second run with the reviewer swapped throughmodel_roles. Nothing is written to the working directory.python main.py --task examples.<task>onmainand on this branch, and compare the output records and metadata.make studio, then open the task list and a few workflows. The YAML, code, sample-data and preview views should behave as before.🖼️ Screenshots (if applicable)
How a host application embeds SyGra after this PR:
flowchart LR subgraph Host["🏠 Host application (web service, worker, notebook)"] Rows["rows in memory"] Client["own LLM client / gateway"] Result["RunResult: records, failures, usage, cost"] end subgraph SyGra["🧩 SyGra (library mode)"] Cfg["Workflow.from_config(dict)\n+ hook objects + model roles"] Run["await workflow.arun()\nper-run RunContext"] Graph["graph engine\n(LLM, lambda, sampler, edges)"] end Rows --> Cfg --> Run --> Graph Graph -- "LLMRequest" --> Client Client -- "ModelResponse + usage" --> Graph Graph --> Result✅ Checklist
main.pyruns against the previous commit for every item; Studio API smoke test on 3.11 and 3.13; the example run against an installed wheel)📝 Notes
Workflow.run()raisesModelServerDownErrorinstead of exiting, and no longer loads.envimplicitly (passenv_file=".env").RunResult, alistsubclass. YAML tasks run throughWorkflownow return their records instead ofNone.id.ConfigurationErroris raised as is, not wrapped inExecutionError.pandarallel, which is declared but never imported by SyGra, doesn't import under pandas 3.docs/features/external_llm_providers.md;docs/library/sygra_library.md(async execution, run results,from_config, hooks, memory sources,write_output, model roles),docs/library/sygra_library_examples.md,docs/getting_started/model_configuration.mdanddocs/features/metadata_tracking.md.model_type: externalyet.post_processandtoolsstill take dotted paths only.logs/out.logis still created in the working directory when SyGra is imported.messagesoutput key isn't JSON-serializable in programmatic runs, so use a plain output key.